Communication Theory II 

Lecture 4: Review on Fourier analysis and probabilty theory 
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Course Website 

o http://lms.mans.edu.eg/eng/ 

o The site contains the lectures, quizzes, homework, and open 
forums for feedback and questions 

o Log in using your name and password 

o Password for quizzes: third 

o One page quiz: for less download time 



Lecture Outlines 



o Review on Fourier analysis of signals and systems 

■ The Dirac delta function 

■ Fourier transform of periodic signals 

■ Transmission of signals through LTI systems 

■ Hilbert transform 
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The Dirac Delta Function (Unit Impulse) 



Dirac delta function: 
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o An even function of time t, centered at the origin t = 0 

o Sifting property: sifts out the value g(tO) of the function g{t) at time t = to, 

OO 

where J gW< 5(r-r 0 )dr = g(f 0 ) 



o Replication property: convolution of any function with the delta function 

OO 

leaves that function unchanged J g( T)S(t- T)dr = g(t) 



— OO 



The Dirac Delta Function (cont'd) 




The Dirac 



The delta function may be viewed 
as the limiting form of a pulse of 
unit area (symmetric with respect to 
the origin) as the duration of the 
pulse approaches zero 



Function (cont'd) 



S(t) = lim I rect ( f - , 
Rectangular impulse 



8(t) 



S(t) = lim 2Wsinc(2Wt) 

W — > oo 



Sine impulse 
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Existence of Fourier Transform 



For the Fourier transform of a signal git) to exist, it is sufficient but not necessary that 
the nonperiodic signal g(t) satisfies three Ditichlet's conditions of its own: 

1 . The function g(t) is single valued, with a finite number of maxima and minima in 
any finite time interval. 

The function g(t) has a finite number of discontinuities in any finite time interval. 
3. The function g(t) is absolutely integrable; that is. 



o Physical readability is a sufficient condition for the existence of a Fourier 
transform (e.g., all energy signals are Fourier transformable ). 



o What about power and periodic signals? Do they have a Fourier transform? 




Condition for energy signal: 
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Fourier Transform of Periodic Signals 



o Can Fourier transform works for periodic or power signals? 



Periodic signal (power) 




gT0(O ? 
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Time shift property 



8(t) = 



gT (t), -^<t<^ 



0, 



2 ~ 2 
elsewhere 



Aperiodic signal with computable G(r) 



i 


g(f)=A«/x)-G(r) 
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m = -oo 



- G T (f) = G(f) £ exp(-j2jiro/T 0 ). 



-OO 
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Fourier Transform for Analyzing Signals and Systems 



o Provides the mathematical link between the time domain of a signal 
(waveform) and its frequency domain (spectrum) 

o Time and frequency response of a linear time-invariant (LTI) system 
defined in terms of its impulse response and frequency response, 

respectively Time domain 



x(t) 
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y(t) = h(t)*x(t) 
t 

I 

Y(/) = H(/).G(/) 



Frequency domain 



If x^n)^ 



h(n) 



■» yi(n) 



and x 2 (n) 



h(n) 



■» y 2 (n) 



Then a 1 y 1 (n)+a 2 y 2 (n)=H{a 1 x 1 (n)+a 2 x 



Where al, a2 are scalars 



(a) 



A (a) linear and (b) a time invariant system 



If x,(n) -» 


h(n) 


Yi(n) 


Then x^n-k) -> 


h(n) 


■» y,(n-k) 



(b) 
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Hi I bert Transform 



o A quadrature filter: phase angles of all components of a given signal are 
shifted by ±90° 

Hilbert transform pair 



oo 

*0 = J _ 

— oo 



t-T 



git) = 



-If ^dr 

7rJ r 



o Example: g(t)=5(f) 



— oo 



t- T 



dr = 



Kt 



Using replication property of the delta function 
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Hilbert-transform pairs 



Time function 



1. m{t)cos(2itf c t) 

2. m(r)sin(2# c f) 

3. cos(27&f) 

4. s\n(2nf c t) 



5. 



siiw 



Hilbert transform 



/n(7)sitl( [Tjtfj) >Mf) is band limited to the interval -W <f<W where IV </ c , 

-fff(0cos(2n£f) m(0 is band limited to the interval -W </< W, where W < / c , 
sin(27£f) 

-cos(2jt/ c 0 
I - cos r 



6. rect(f) 



7. m 



-i]n 
n 



JL 
jet 



t- 1/2 



r + 1/2 



8. 



i +r 



l + r 



9. ± 



Notes: S(t) denotes Dirac delta function; rect(/) denotes rectangular; function: In denotes natural logarithm, 
* In the first two pairs, it is assumed that m(t) is band limited to the interval - W <f< W, where W <f c . 
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Hilbert transform (cont'd) 

1 

oThe function g(t) may be interpreted as g(t)* — ^ G (/) . -j sgn (/) 

1 



g(t) 




9= 


9 


1 




G(/) * 


H(/) 




-j sgn (/) 



g(0 = h(t)*g(t) 
t 

I 

► G(/)= H{/).G{/) 



s g n C/) = j 0, /= 0 
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Hilbert Transform properties 



1. 



\G(f)\ = \G(f)\ 



2. arg[G(/)] = -arg{G(/)} 



3. A Hilbert pair is orthogonal over the entire time interval 



CO 



g(t)g(t)dt = 0 



— CO 
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Lecture Outlines 



o Review on probability theory* 

■ Deterministic vs. probabilistic mathematical models 

■ Probability theory, random variables, and the distribution functions 

■ The concept of expectation and second order statistics 

■ Characteristic function, the center limit theory and the Bayesian interface 
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*Note that soft materials are from COMPE 467 - Pattern Recognition, www.atilim.edu.tr/~mkahraman/2_Foundations.ppt 



Deterministic vs. Probabilistic Models 

o Deterministic mathematical model: if there is no uncertainty 
about its time-dependent behavior at any instant of time 

o Example: linear time-invariant systems 

o Problem: underlying physical phenomenon involves too many 
unknown factors (e.g., noise, fading, interference) 

o Probabilistic model accounts uncertainty in mathematical terms 

o Probabilistic models are intended to assign probabilities to the 
collections (sets) of possible outcomes of random experiments 



Probability Theory: Sets 



o Probability makes extensive use of set operations 

o A set is a collection of objects, which are the elements of the set 

o If S is a set and x is an element of S, we write x6S, otherwise x $ S 

o A set can have no elements, in which case it is called the empty set, 
denoted by 0 

o The set of all possible outcomes of an experiment is the sample 
space or universe, denoted Q 

o An event A is a (set of) possible outcomes of the experiment, and 
corresponds to a subset of Q 



Probability Theory: Sets (cont'd) 



o Sets can be specified as 
o For example, 

■ The set of possible outcomes of a die roll is {1, 2, 3, 4, 5, 6}, 

■ The set of possible outcomes of a coin toss is {H, T}, where H stands for 
"heads" and T stands for "tails" 
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Probability Theory: Sets (cont'd) 



o Alternatively, we can consider the set of all x that have a certain property P, and 
denote it by 

{x | x satisfies P). 
(The symbol "|" is to be read as "such that/') 

o For example, 

• the set of even integers can be written as {k \ k/2 is integer}. 
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Probability Theory: Sets Operations 



o Complement: 

• The complement of a set S, with respect to the universe Q, is the set {x E Q |x ^ S} of i 
elements of Q that do not belong to 5, and is denoted by S c . 

o Union: 



SUT = {x\xe S or xeT}, 



o Intersection: 





T is a subset of S 

res 



SnT = {x\x eS and x eT] 




Probability Theory: Sets Operations 



o Disjoint sets: 

■ two sets are said to be disjoint if their intersection is empty 

o Partition of a set A: 

■ A collection of disjoint subsets of A, where their union is >A 




Partition of Q 




Probability measure 



o A probability law / measure is a function p(>4) that assigns a nonnegative 
value to event A 

■ based on the expected proportion of number of times that event A is actually likely 
to happen: encodes our belief in the likelihood event A occurring when the 
experiment is conducted 
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Figure 3> Illustration of the relationship between sample space, events, and probability 
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Probability Theory: Axioms of Probability 



Probability Axioms 


1. 


(Nonnegativity) P(A) > Q f for every event A. 


2. 


( Adclitivitv^ If A and H are two disioint events, then the orobabilitv 




of their union satisfies 




P(AUB) = P(A)+P(B). 




More generally, if the sample space has an infinite number of elements 




and A\ , A 2 , . . . is a sequence of disjoint events, then the probability of 




their union satisfies 




P(AiUA 2 U ■■■) = P(Ai)+P(A 2 )-\ . 


3. 


(Normalization) The probability of the entire sample space SI is 




equal to 1, that is, P(fi) = 1. 
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Probability Theory: Axioms of Probability (cont'd) 

oThe probability function P(A) must satisfy the following: 



0<P(4)<1, 
P(Q) = X^(A) 

A l nA J =j=> P(A i u A . ) = P{A i ) + P(A . ), 
A^,^=> p ( A u A, ) = P(4 ) + P(A,. ) - P( A n A, ), 
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Example 1 



Example Consider an experiment involving a single coin toss. There are two 
possible outcomes, heads (H) and tails (T). The sample space is Q = { H,T}> and 
the events are 

{HX}, {T}, 0. 

If the coin is fair, i.e., if we believe that heads and tails are ''equally likely," we 
should assign equal probabilities to the two possible outcomes and specify that 
P({H}) = P({T}) = 0.5. 
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Example 1 (cont'd) 



The additivity axiom implies that 

p({ff,r}) = p({tf}) + p({T}) = i ; 

which is consistent with the normalization axiom. Thus, the probability law is given 

by 

P({HJ}) = 1, P({if})=0.5, P(\T})=te, P(0) = O, 
and satisfies all three axioms. 



Example 2 



Consider smother experiment involving three coin tosses. The outcome i 
now be a 3-long string of heads or tails. The sample space is 

fl = |M ; HHT, HTH, HTT, THH, THT, TTH, TTT}. 

We assume that each possible outcome has the same probability of 1/8. 



Example 2 (cont'd) 



Consider, as an example, the event 

A = {exactly 2 heads occur} = {HHT, HTH, THE). 

Using additivity, the probability of A is the sum of the probabilities of its elements: 

P({HHT, HTH, THH}) = P({HHT}) + P({HTH }) -hP {{THH}) 

8 8 8 
_ 3 

" 8" 
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o Real experiments involve using one 
or more real valued quantities called 
random variables 

oThe outcomes are associated with 
some numerical values of interest 
(random variable). 

o Example: if the experiment is the 
selection of students from a given 
population, we may wish to consider 
their grade point average. 



The random variable is a 

function whose domain is a 
sample space and whose range 
is some set of real numbers 



Variables 



Random 
variable 00 

A 



experiment 




•txi 



Probability 

T 1 



1 0 



Illustration of the relationship between sample 
space, random variables, and probability. 30 



Discrete and Continuous Random Variables 

• Random variables can discrete, e.g., the number of heads in three 
consecutive coin tosses, or continuous, the weight of a class member. 
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Questions 
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